Storage medium having stored therein information processing program, information processing apparatus, information processing system, and information processing method

ABSTRACT

An image including at least a part of a virtual world is generated as a first display image, a first sound created in the virtual world is generated, and a second sound forming a part of the first sound is selected in accordance with a user operation. Then, control is performed such that the first display image is displayed on a first display apparatus. Control is performed such that the generated first sound is output from a first loudspeaker in synchronization with the first display image displayed on the first display apparatus. Control is performed such that the selected second sound is output from a second loudspeaker.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2012-251896, filed onNov. 16, 2012, is incorporated herein by reference.

FIELD

The technology shown here relates to a storage medium having storedtherein an information processing program, an information processingapparatus, an information processing system, and an informationprocessing method, and in particular, relates to a storage medium havingstored therein an information processing program for, for example,outputting a sound from a loudspeaker, and an information processingapparatus, an information processing system, and an informationprocessing method for, for example, outputting a sound from aloudspeaker.

BACKGROUND AND SUMMARY

Conventionally, there is a game apparatus that outputs the sound ofperforming one composition as an ensemble using a plurality ofinstruments. In the game apparatus, a user selects a part to be assignedto the user themselves. When the selection of the assigned part iscompleted, the ensemble is started. Then, the user can participate inthe ensemble by performing an input operation based on the assignedpart.

The game apparatus, however, can merely provide the audio output of thecomposition performed as an ensemble from, for example, a pair ofloudspeakers provided in a monitor. Thus, even a composition performedas an ensemble may lack variety, as compared to other types of soundoutputs.

Therefore, it is an object of an exemplary embodiment to provide astorage medium having stored therein an information processing programcapable of a wide variety of sound outputs, and an informationprocessing apparatus, an information processing system, and aninformation processing method that are capable of a wide variety ofsound outputs.

To achieve the above object, the exemplary embodiment can employ, forexample, the following configurations. It should be noted that it isunderstood that, to interpret the descriptions of the claims, the scopeof the claims should be interpreted only by the descriptions of theclaims. If there is a conflict between the descriptions of the claimsand the descriptions of the specification, the descriptions of theclaims take precedence.

An exemplary configuration of a non-transitory computer-readable storagemedium having stored therein an information processing program accordingto the exemplary embodiment is executed by a computer of an informationprocessing apparatus for outputting an image and a sound to a firstdisplay apparatus having a first loudspeaker and for outputting at leasta sound to another apparatus having a second loudspeaker. Theinformation processing program causes the computer to execute:generating an image including at least a part of a virtual world as afirst display image; generating a first sound created in the virtualworld; selecting, in accordance with a user operation, a second soundforming a part of the first sound; performing control such that thefirst display image is displayed on the first display apparatus;performing control such that the generated first sound is output fromthe first loudspeaker in synchronization with the first display imagedisplayed on the first display apparatus; and performing control suchthat the selected second sound is output from the second loudspeaker.

On the basis of the above, in accordance with a user operation, it ispossible to select the second sound forming a part of the first soundand listen to the selected second sound using another apparatus. Thisenables a wide variety of sound outputs.

In addition, in the generation of the first sound, if the second soundhas been selected, the first sound may be newly generated by removingthe second sound from the first sound.

On the basis of the above, it is possible to listen to a sound obtainedby removing the second sound from the first sound, using the firstdisplay apparatus. This enables a wider variety of sound outputs.

In addition, in the generation of the first display image, an image inwhich a plurality of objects are placed in the virtual world may begenerated as the first display image. The information processing programmay further cause the computer to execute selecting at least one objectfrom among the plurality of objects in accordance with a user operation.In this case, in the generation of the first sound, the first sound maybe generated in accordance with the objects displayed in the firstdisplay image. In the selection of the second sound, the second soundmay be selected in accordance with the at least one selected object.

On the basis of the above, the selection of at least one object fromamong the plurality of objects displayed on the first display apparatusprovides the audio output of a sound based on the at least one selectedobject from the second loudspeaker. This enables the operation ofselecting an object to select a sound based on the selected object andlisten to the sound from the second loudspeaker.

In addition, in the generation of the first display image, an imageobtained by removing the at least one selected object from the pluralityof objects and placing the at least one remaining object may begenerated as the first display image. In the generation of the firstsound, a second sound based on the at least one selected object may beremoved from the first sound based on the plurality of objects to newlygenerate a sound based on the at least one remaining object as the firstsound.

On the basis of the above, the selection of at least one object fromamong the plurality of objects displayed on the first display apparatusprovides from the second loudspeaker the audio output of a sound basedon the at least one selected object, and also provides from the firstloudspeaker the audio output of the first sound obtained by removing thesound based on the at least one selected object. This enables theoperation of selecting an object to move the audio output based on theselected object from the first loudspeaker to the second loudspeaker.

In addition, the other apparatus may be a second display apparatushaving a display screen. The information processing program may furthercause the computer to execute: generating an image representing the atleast one selected object, as a second display image; and performingcontrol such that the second display image is displayed on the seconddisplay apparatus. In this case, in the selection of the second sound,the second sound may be selected in accordance with the at least oneobject displayed as the second display image.

On the basis of the above, at least one of the plurality of objectsdisplayed on the first display apparatus is displayed on the seconddisplay apparatus, thereby providing from the second display apparatusthe audio output of a sound based on the at least one displayed object.This enables the operation of selecting an image and a sound to view theselected image and listen to the selected sound, using the seconddisplay apparatus.

In addition, in the generation of the first sound, a sound of which asound source is the at least one object displayed as the first displayimage may be generated as the first sound. In the selection of thesecond sound, a sound of which a sound source is the at least one objectdisplayed as the second display image may be selected as the secondsound.

On the basis of the above, the display of an object serving as the soundsource of a sound on the second display apparatus provides from thesecond display apparatus the audio output of a sound of which the soundsource is the displayed object. This enables the operation of selectinga sound source to view the image and listen to the sound, using thesecond display apparatus.

In addition, the first display apparatus may be a stationary displayapparatus. The second display apparatus may be a portable displayapparatus.

On the basis of the above, it is possible to display on the stationarydisplay apparatus the virtual world in which the plurality of objectsare placed, and provide from the stationary display apparatus the audiooutput of a sound based on the plurality of objects, while displaying onthe portable display apparatus a virtual world in which the selectedobject is placed, and providing from the portable display apparatus theaudio output of a sound based on the selected object.

In addition, the second display apparatus may include an operation inputunit configured to output data based on a user operation input. In thiscase, the information processing program may further cause the computerto execute: acquiring the data output from the operation input unit;controlling, in accordance with the acquired data, the at least oneobject displayed as the second display image; and changing, inaccordance with the acquired data, at least one of a volume, a tone, astep, and the number of produced sounds of the second sound.

On the basis of the above, it is possible to change the second sound ofwhich the audio output is provided from the second display apparatus,and listen to the changed second sound.

In addition, in the generation of the first sound, a sound of performingone composition as an ensemble using a plurality of instruments in thevirtual world may be generated as the first sound. In the selection ofthe second sound, a sound of performing the composition with at leastone instrument selected from among the plurality of instruments inaccordance with a user operation may be generated as the second sound.In the control of outputting the second sound, the second sound ofperforming the composition may be output from the second loudspeaker insynchronization with the performance of the composition output as thefirst sound from the first loudspeaker.

On the basis of the above, it is possible to extract the sound ofplaying at least one instrument from the sound of performing as anensemble using a plurality of instruments, and listen to the extractedsound.

In addition, the information processing program may further cause thecomputer to execute determining, in accordance with a user operation, aposition and/or a direction indicated in the virtual world displayed asthe first display image. In this case, in the generation of the firstsound, an entire sound created in an entirety of the virtual worlddisplayed as the first display image may be generated as the firstsound. In the selection of the second sound, on the basis of theposition and/or the direction indicated in the virtual world, a soundcreated in a periphery of the position indicated in the virtual worldand/or a sound created in a predetermined range with respect to thedirection indicated in the virtual world may be generated and selectedas the second sound.

On the basis of the above, it is possible to provide from the firstloudspeaker the audio output of the entire sound created in the entiretyof the virtual world, and also provide from the second loudspeaker theaudio output of a sound based on the position and/or the directionindicated in the virtual world.

In addition, the exemplary embodiment may be carried out in the forms ofan information processing apparatus and an information processing systemthat include units configured to perform the above operations, and aninformation processing method including the above operations.

On the basis of the exemplary embodiment, in accordance with a useroperation, it is possible to select a sound forming a part of aparticular sound to listen to the selected sound using anotherapparatus. This enables a wide variety of sound outputs.

These and other objects, features, aspects and advantages of theexemplary embodiments will become more apparent from the followingdetailed description of the exemplary embodiments when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a non-limiting example of aninformation processing system 1;

FIG. 2 is a diagram showing non-limiting examples of images displayed inthe information processing system 1;

FIG. 3 is a diagram showing non-limiting examples of an entire image ofa virtual world displayed on a monitor 4 and a sound output from themonitor 4;

FIG. 4 is a diagram showing non-limiting examples of a display imagedisplayed on a terminal apparatus 2 and an output sound output from theterminal apparatus 2 and non-limiting examples of a display imagedisplayed on the monitor 4 and an output sound output from the monitor 4when a selection object has been selected;

FIG. 5 is a diagram showing a non-limiting example where the selectionobject and the output sound are controlled by an operation on theterminal apparatus 2;

FIG. 6 is a diagram showing non-limiting examples of main data andprograms stored in a memory 6 of an information processing apparatus 3;

FIG. 7 is a diagram showing a non-limiting example of composition dataDc;

FIG. 8 is a flow chart showing a non-limiting example of the first halfof the information processing performed by the information processingapparatus 3; and

FIG. 9 is a flow chart showing a non-limiting example of the second halfof the information processing performed by the information processingapparatus 3.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS

With reference to FIG. 1, a description is given of an informationprocessing apparatus for executing an information processing program andan information processing system including the information processingapparatus, according to an exemplary embodiment. It should be noted thatFIG. 1 is a block diagram showing an example of an informationprocessing system 1 including an information processing apparatus 3. Asan example, the information processing apparatus 3 is composed of astationary game apparatus, and the information processing system 1 iscomposed of a game system including the stationary game apparatus.

In FIG. 1, the information processing system 1 includes a terminalapparatus 2, the information processing apparatus 3, and a monitor 4.The information processing system 1 according to the exemplaryembodiment generates images (for example, game images) and displays theimages on display apparatuses (the terminal apparatus 2 and the monitor4).

In the information processing system 1, the information processingapparatus 3 executes information processing in accordance with an inputto the terminal apparatus 2, and the images obtained as a result of theexecution of the information processing are displayed on the terminalapparatus 2 and/or the monitor 4. As described above, in the exemplaryembodiment, the information processing system 1 is configured to achievean input function, an information processing function, and a displayfunction by a plurality of apparatuses. It should be noted that inanother exemplary embodiment, the information processing system 1 may becomposed of a single information processing apparatus (for example, aportable information processing apparatus) having these functions.

The terminal apparatus 2 is an input apparatus that can be held by auser (a portable input apparatus). The terminal apparatus 2 is capableof communicating with the information processing apparatus 3. Theterminal apparatus 2 transmits operation data representing the operationon the terminal apparatus 2 to the information processing apparatus 3.Further, in the exemplary embodiment, the terminal apparatus 2 includesa display section (an LCD 11), and therefore, the terminal apparatus 2serves also as a display apparatus. When an image has been transmittedfrom the information processing apparatus 3, the terminal apparatus 2displays the image on the LCD 11.

In addition, the terminal apparatus 2 includes a loudspeaker 12, andtherefore, the terminal apparatus 2 serves also as a display apparatusand the audio output apparatus. The loudspeaker 12 may be, for example,a pair of stereo loudspeakers, and the output of the loudspeaker 12 maybe controlled by a sound IC, which is a circuit for controlling theoutput of audio data. When a sound has been transmitted from theinformation processing apparatus 3, the terminal apparatus 2 outputs thesound from the loudspeaker 12 through the sound IC.

In addition, the terminal apparatus 2 includes an operation section 13as an input section. As an example, the operation section 13 includes atouch panel, which detects the position of an input provided to apredetermined input surface (for example, the screen of the displaysection) provided in a housing, an analog stick, a directional pad, anoperation button, and the like. Further, the terminal apparatus 2 mayinclude an acceleration sensor, a gyro sensor, and the like as an inputsection. The acceleration sensor and the gyro sensor detect theacceleration and the angular velocity of the terminal apparatus 2 withrespect to a predetermined axial direction.

The information processing apparatus 3 performs various types ofinformation processing performed in the information processing system 1,such as the process of generating an image. In the exemplary embodiment,the information processing apparatus 3 has a CPU (control section) 5 anda memory 6. The CPU 5 executes a predetermined information processingprogram using the memory 6, thereby achieving various functions of theinformation processing apparatus 3. It should be noted that theinformation processing apparatus 3 may have any configuration so long asit can perform the information processing described above. In theexemplary embodiment, the information processing apparatus 3 generatesimages (for example, game images) and outputs the generated images tothe terminal apparatus 2 and the monitor 4, which serve as displayapparatuses.

The monitor 4 is an example of a display apparatus that displays agenerated image and the audio output apparatus that outputs a generatedsound. The monitor 4 is capable of receiving data transmitted from theinformation processing apparatus 3. When an image and a sound generatedby the information processing apparatus 3 have been transmitted to themonitor 4, the monitor 4 displays the image, and also outputs the soundfrom the loudspeaker 41.

Next, with reference to FIGS. 2 through 5, a description is given of anoverview of the information processing performed by the informationprocessing apparatus 3, before the description of specific processingperformed by the information processing apparatus 3. FIG. 2 is a diagramshowing examples of images displayed in the information processingsystem 1. FIG. 3 is a diagram showing examples of an entire image of avirtual world displayed on the monitor 4 and a sound output from themonitor 4. FIG. 4 is a diagram showing examples of a display imagedisplayed on the terminal apparatus 2 and an output sound output fromthe terminal apparatus 2 and examples of a display image displayed onthe monitor 4 and an output sound output from the monitor 4 when aselection object has been selected. FIG. 5 is a diagram showing anexample where the selection object and the output sound are controlledby an operation on the terminal apparatus 2.

As shown in FIG. 2, in the exemplary embodiment, as an example, a gameimage is displayed on each of the terminal apparatus 2 and the monitor4. For example, the user operates the terminal apparatus 2 while holdingit, thereby selecting an object placed in a virtual world displayed onthe monitor 4. Thus, the user can cause the selected object to bedisplayed on the LCD 11 of the terminal apparatus 2.

As shown in FIGS. 2 and 3, for example, an entire image of the virtualworld is displayed on the monitor 4. For example, a plurality of objectsOBJ (five objects OBJ1 through OBJ5 in FIG. 3) are displayed in theentire image. Then, all sounds (an entire sound) created in the virtualworld are output from the loudspeaker 41 of the monitor 4 such that theplurality of objects OBJ are the sound sources of the respective sounds.As an example, the plurality of objects OBJ perform the same compositionusing respective instruments in the virtual world, so that the audiooutput of the composition performed as an ensemble by the plurality ofobjects OBJ (that is, the sound into which all the sounds of the objectsOBJ performing the same composition are combined) is provided as theentire sound.

Here, the user operates the terminal apparatus 2 (for example, using theoperation section 13), and thereby can select an object OBJ from amongthe plurality of objects OBJ displayed on the monitor 4 (the objectselected by the user will hereinafter be referred to as a “selectionobject”). It should be noted that one selection object may be selected,or a plurality of selection objects may be selected, from among theplurality of objects OBJ displayed on the monitor 4. For example, in theexample shown in FIG. 4, the user selects the object OBJ1 displayed onthe monitor 4 as a selection object. Then, if the selection object OBJ1has been selected, the selection object OBJ1 is erased from the entireimage displayed on the monitor 4, and the selection object OBJ1 isdisplayed on the LCD 11 of the terminal apparatus 2. The display of theselection object is thus controlled, thereby enabling a representationas if the selection object has moved from the virtual world displayed onthe monitor 4 to a virtual world displayed on the terminal apparatus 2.

At this time, from the loudspeaker 12 of the terminal apparatus 2, theaudio output of the sound (a selection sound) which is created in thevirtual world and of which the sound source is the object displayed onthe LCD 11 (that is, the selection object) is provided. For example, inthe example shown in FIG. 4, the selection object OBJ1 is displayed onthe LCD 11, and therefore, the audio output of the sound of theselection object OBJ1 performing the composition is provided from theloudspeaker 12. That is, the user performs the operation of selectingthe selection object, thereby providing, as the selection sound from theterminal apparatus 2, the audio output of the sound forming a part ofthe entire sound of which the audio output has been provided from themonitor 4. Thus, if the entire sound of which the audio output isprovided from the monitor 4 includes a sound that the user wishes toview and listen to in a focused manner, the user can extract the soundto provide the audio output of the extracted sound from the terminalapparatus 2. Thus, it is possible to provide an unconventional viewingand listening environment to the user.

In addition, if the selection object OBJ1 has been selected from amongthe plurality of objects OBJ displayed on the monitor 4, a new entiresound is generated by removing the sound of which the sound source isthe selection object OBJ1, from the entire sound of which the audiooutput has been provided from the loudspeaker 41 before the selection ofthe selection object OBJ1. Then, the audio output of the new entiresound is provided from the loudspeaker 41. For example, in the exampleshown in FIG. 4, a sound obtained by removing the selection sound of theselection object OBJ1 performing from the entire sound generated by theplurality of objects OBJ1 through OBJ5 performing as an ensemble, thatis, the sound generated by the plurality of objects OBJ2 through OBJ5performing as an ensemble, is newly generated as the entire sound. Then,the audio output of the new entire sound is provided from theloudspeaker 41. Thus, if the entire sound of which the audio output isprovided from the monitor 4 includes a sound that the user wishes toremove, the user can remove the sound to provide the audio output of theresulting entire sound from the monitor 4. Thus, it is possible toprovide an unconventional viewing and listening environment to the user.

In addition, the user operates the terminal apparatus 2 with theselection object displayed on the LCD 11, and thereby can control thestate, the action, and the like of the displayed selection object. Forexample, in the example shown in FIG. 5, the user operates the operationsection 13 (for example, an L button or a R button) with the selectionobject displayed on the LCD 11, thereby changing the instrument playedby the selection object OBJ1. Thus, the selection sound of which theaudio output has been provided from the loudspeaker 12 of the terminalapparatus 2 (that is, the sound of the selection object OBJ1 playing theinstrument before the change) changes to a sound based on the change ofinstruments and different in tone (timbre) (that is, the sound of theselection object OBJ1 playing the instrument after the change) even inthe same composition. This enables the user to provide the audio outputof a desired sound. It should be noted that an operation on theselection object may not only enable a change of instruments, but alsoenable changes in other states and other actions of the selectionobject. For example, the user may operate the terminal apparatus 2,thereby changing another device (for example, a device for adding asound effect to playing or singing, such as an effect unit or an echounit) used by the selection object. Alternatively, the performance perse of the selection object may be given in accordance with the aboveoperation. Yet alternatively, the action of the performance of theselection object may be changed (for example, the performance strength,the performance technique, or the performance excellence of theselection object may be changed). In this case, not only the tone butalso at least one of the tone, the volume, the step, and the number ofproduced sounds of the selection sound of which the audio output hasbeen provided from the loudspeaker 12 changes in accordance with theabove operation.

It should be noted that the above description is given using the examplewhere the selection object is displayed on the terminal apparatus 2.Alternatively, the selection object may not need to be displayed on theterminal apparatus 2. In this case, as a first example, the selectionobject is not displayed on the terminal apparatus 2, and only the audiooutput of the selection sound is provided from the loudspeaker 12.Meanwhile, the entire image obtained by erasing the selection object isdisplayed on the monitor 4, and the audio output of the entire soundobtained by removing the selection sound is provided from theloudspeaker 41. As a second example, the selection object is notdisplayed on the terminal apparatus 2, and only the audio output of theselection sound is provided from the loudspeaker 12. Meanwhile, theentire image before the selection of the selection object is displayedas it is on the monitor 4 without erasing the selection object, and theaudio output of the entire sound before the selection of the selectionobject is provided as it is from the loudspeaker 41. In either case, itis possible to extract a partial sound from the entire sound, andprovide the audio output of the extracted sound from the terminalapparatus 2. This makes it possible to achieve a part of the aboveeffect.

In addition, the selection object displayed on the terminal apparatus 2may be returned to and displayed again on the monitor 4 in accordancewith a predetermined operation. For example, the selection object iserased from the game image displayed on the terminal apparatus 2, andthe selection object is added to the entire image on the monitor 4. Inthis case, a new entire sound is generated by adding the sound of whichthe sound source is the selection object, to the entire sound of whichthe audio output has been provided from the loudspeaker 41 before theadditional display of the selection object in the previous entire image.Then, the audio output of the new entire sound is provided from theloudspeaker 41. Here, as described above, in accordance with anoperation on the terminal apparatus 2, it is possible to change at leastone of the tone, the volume, the step, and the number of produced soundsof the selection sound of the selection object performing. Consequently,the newly generated entire sound is a sound subjected to this change, ascompared to the entire sound before the selection of the selectionobject.

In addition, the above description is given using the example where atleast one of the plurality of objects placed as the sound sources isselected, thereby providing from the terminal apparatus 2 the audiooutput of the sound of which the sound source is the selection object.Alternatively, the audio output of the selection sound may be providedfrom the terminal apparatus 2 in another form. For example, a position(an indicated position) and/or a direction (an indicated direction) inthe virtual world displayed on the monitor 4 are set in accordance withan operation on the terminal apparatus 2. Then, in the entire soundwhich is created in the virtual world and of which the audio output isprovided from the monitor 4, the audio output of a sound, of the virtualworld, created in the periphery of the indicated position set in thevirtual world (for example, a sound from a sound source present in apredetermined range such that the indicated position is a reference), orthe audio output of a sound, of the virtual world, created in theindicated direction set in the virtual world (for example, a sound froma sound source present in a predetermined range such that the indicateddirection is a center), is provided from the terminal apparatus 2.Specifically, if an object placed in the virtual world displayed on themonitor 4 has been selected in accordance with an operation on theterminal apparatus 2, the audio output of a sound created in theperiphery of the object, or the audio output of a sound created in apredetermined range such that the forward direction of the object is areference, is provided from the terminal apparatus 2. Also when theindicated position and/or the indicated direction are thus set in thevirtual world displayed on the monitor 4, it is possible to extract apartial sound from the entire sound of which the audio output isprovided from the monitor 4, and provide the audio output of theextracted sound from the terminal apparatus 2.

Next, a detailed description is given of the information processing (forexample, game processing) performed by the information processingapparatus 3. It should be noted that the information processing isdescribed below using an example where a selection object selected fromamong a plurality of objects displayed on the monitor 4 is displayed onthe terminal apparatus 2, and the audio output of the sound ofperforming the same composition is provided from each of the monitor 4and the terminal apparatus 2 (see FIGS. 2 through 5). First, withreference to FIGS. 6 and 7, a description is given of main data used inthe information processing. It should be noted that FIG. 6 is a diagramshowing examples of main data and programs stored in the memory 6 of theinformation processing apparatus 3. FIG. 7 is a diagram showing anexample of composition data Dc.

As shown in FIG. 6, the following are stored in the data storage area ofthe memory 6: terminal operation data Da; operation object position dataDb; composition data Dc; first sound data Dd; second sound data De;display image data Df; and the like. It should be noted that the memory6 may store, as well as the data shown in FIG. 6, data and the likenecessary for the information processing, such as data used in anapplication to be executed. Further, in the program storage area of thememory 6, various programs Pa included in the information processingprogram are stored.

The terminal operation data Da has stored therein a series of pieces ofoperation information (terminal operation data) transmitted astransmission data from the terminal apparatus 2, and is updated to thelatest terminal operation data. For example, the terminal operation dataDa includes operation input data Da1 and the like. The operation inputdata Da1 is data representing the content of the operation on theoperation section 13.

The object data Db is data for setting objects to be displayed on theterminal apparatus 2 and the monitor 4. For example, the object data Dbincludes display position data Db1, performance instrument data Db2, andthe like. The display position data Db1 is data representing the displayposition of each object displayed on the terminal apparatus 2 and/or themonitor 4. The performance instrument data Db2 is data representing thetype of instrument and the performance method (the type of a device foradding a sound effect to playing or singing, the performance strength,the performance technique, the performance excellence, and the like)used for performance by each object displayed on the terminal apparatus2 and/or the monitor 4.

The composition data Dc is configured in, for example, a stream format,and includes data representing a plurality of compositions of which theaudio outputs are provided from the terminal apparatus 2 and the monitor4. For example, as shown in FIG. 7, the composition data Dc isconfigured such that a plurality of instruments are assigned to aplurality of streams so that the performance of the same composition bya plurality of instruments can be selectively controlled by specifyinginstruments by the stream numbers assigned to the respectiveinstruments. That is, the composition data Dc is configured such thatone stream is assigned to one part (one instrument) in a musicalperformance. For example, the composition data Dc is used to perform thesame composition as an ensemble with a plurality of instruments, using asound source (a DSP (digital signal processor), an ARAM (audio RAM), andthe like) provided in the information processing apparatus 3. The soundsource superimposes performance streams having the specified streamnumbers to output a musical sound. Then, the sound source produces themusical sound having the steps, the tones, and the volumes specified onthe basis of the performance parameters indicated by the CPU 5, at aspecified performance tempo.

For example, the composition data Dc has a plurality of performancestreams for each of the prepared compositions. Each performance streamis associated with a stream number and an instrument name. As anexample, in the composition data Dc, different instruments are assignedto stream numbers, and performance streams are described so as tocorrespond to the stream numbers and the instruments. For example, aperformance stream 1 having the instrument name “guitar” is assigned tothe stream number “1”, and a performance stream 2 having the instrumentname “piano” is assigned to the stream number “2”. Then, the indicationof stream numbers and performance timing enables the sound source (theDSP and the ARAM) provided in the information processing apparatus 3 tosuperimpose the performance streams having the indicated stream numbers,thereby providing the audio outputs of the performance streams from theloudspeaker 12 of the terminal apparatus 2 and the loudspeaker 41 of themonitor 4 at the indicated performance timing.

It should be noted that the composition data Dc may prepare performancestreams for each apparatus that provides an audio output (that is, eachof the loudspeaker 12 of the terminal apparatus 2 and the loudspeaker 41of the monitor 4). Further, the composition data Dc may be composed ofdata in another format. For example, the composition data Dc may becomposed of sound source control data in the MIDI format. Specifically,the composition data Dc may be configured such that a plurality ofinstruments are assigned to a plurality of tracks (channels) termed MIDIchannels, so that the performance of the same composition by a pluralityof instruments can be selectively controlled by specifying instrumentsby the track numbers assigned to the respective instruments. Further,the volume of the sound to be output may be controlled to vary betweenthe case where the audio output is provided from the loudspeaker 12 ofthe terminal apparatus 2 and the case where the audio output is providedfrom the loudspeaker 41 of the monitor 4. As an example, if the volumeof the sound of which the audio output is provided from the loudspeaker12 is smaller than the volume of the sound of which the audio output isprovided from the loudspeaker 41, the musical sound may be produced suchthat the volume of the sound of which the audio output is provided fromthe loudspeaker 12 is predetermined times (for example, twice) thevolume of the sound of which the audio output is provided from theloudspeaker 41.

The first sound data Dd is data representing a first sound (a monitorgame sound) of which the audio output is provided from the loudspeaker41 of the monitor 4, and is data representing the performance parametersfor specifying the stream numbers, the steps, the tones, the volumes,and the like of the performance streams of which the audio outputs areto be provided as the first sound. The second sound data De is datarepresenting a second sound (a terminal apparatus game sound) of whichthe audio output is provided from the loudspeaker 12 of the terminalapparatus 2, and is data representing the performance parameters forspecifying the stream numbers, the steps, the tones, the volumes, andthe like of the performance streams of which the audio outputs are to beprovided as the second sound, the performance timing of each performancestream, and the like.

The image data Df includes object image data Df1, background image dataDf2, and the like. The object image data Df1 is data for placing objectsin a virtual world to generate a game image. The background image dataDf2 is data for placing a background in a virtual world to generate agame image.

Next, with reference to FIGS. 8 and 9, detailed descriptions are givenof the information processing performed by the information processingapparatus 3. It should be noted that FIG. 8 is a flow chart showing anexample of the first half of the information processing performed by theinformation processing apparatus 3. FIG. 9 is a flow chart showing anexample of the second half of the information processing performed bythe information processing apparatus 3. Here, in the flow charts shownin FIGS. 8 and 9, descriptions are given mainly of, in the informationprocessing performed by the information processing apparatus 3, theprocess of displaying objects on the terminal apparatus 2 and themonitor 4, and providing the audio output from each of the terminalapparatus 2 and the monitor 4. Detailed descriptions of other processesnot directly related to these processes are omitted.

The CPU 5 initializes the memory 6 and the like, and loads theinformation processing program stored in a non-volatile memory or anoptical disk included in the information processing apparatus 3 into thememory 6. Then, the CPU 5 starts the execution of the informationprocessing program. The flow charts shown in FIGS. 8 and 9 are flowcharts showing the processing performed after the above processes arecompleted.

It should be noted that the processes of all the steps in the flow chartshown in FIGS. 8 and 9 are merely illustrative. Thus, the processingorder of the steps may be changed, or another process may be performedin addition to, and/or instead of, the processes of all the steps, solong as similar results are obtained. Further, in the exemplaryembodiment, descriptions are given on the assumption that the CPU 5performs the processes of all the steps in the flow chart.Alternatively, a processor or a dedicated circuit other than the CPU mayperform the processes of some or all of the steps in the flow chart.

Referring to FIG. 8, the CPU 5 performs an initialization process (step61), and proceeds to the subsequent step. For example, the CPU 5constructs a virtual world in which a plurality of objects are to beplaced and which is to be displayed on the monitor 4 in the followingprocesses, and constructs a virtual world in which a selection object isto be placed and which is to be displayed on the terminal apparatus 2 inthe following processes. Then, the CPU 5 initializes the parameters tobe used in the game processing (for example, initializes the secondsound data De so that the audio output of the second sound is notprovided).

Next, the CPU 5 selects a composition to be performed in a game to beperformed in the following processes (step 62), and proceeds to thesubsequent step. For example, the CPU 5 selects a composition on thebasis of a user operation or a predetermined algorithm, readies aplurality of performance streams (see FIG. 7) prepared for thecomposition, and stores the performance streams in the composition dataDc.

Next, the CPU 5 sets objects to perform the selected composition as anensemble (step 63), and proceeds to the subsequent step. For example,the CPU 5 extracts a plurality of objects (a plurality of instruments)to perform the composition selected in the above step 62 as an ensemble.Then, the CPU 5 sets the positions where the plurality of respectiveextracted objects are to be displayed in the virtual world, and theinstruments to be played by the plurality of respective extractedobjects. Then, the CPU 5 stores data representing these settings in thedisplay position data Db1 and the performance instrument data Db2 withrespect to each of the plurality of objects.

Next, the CPU 5 selects performance streams to be employed for a firstsound (step 64), and proceeds to the subsequent step. For example, theCPU 5 sets the stream numbers corresponding to the instruments to beplayed by the respective objects set in the above step 63, as the streamnumbers of the performance streams of which the audio outputs are to beprovided as the first sound. Then, the CPU 5 stores data representingthese settings in the first sound data Dd. Further, the CPU 5 sets theperformance parameters of each performance stream for specifying thesteps, the tones, the volumes, and the like of the performance streamsof which the audio outputs are to be provided as the first sound, andthe performance timing of each performance stream, to the default. Then,the CPU 5 stores data representing these settings in the first sounddata Dd.

Next, the CPU 5 acquires operation data from the terminal apparatus 2,updates the terminal operation data Da (step 65), and proceeds to thesubsequent step. For example, the CPU 5 updates the operation input dataDa1 using data representing the content of the operation on theoperation section 13.

Next, the CPU 5 determines whether or not the operation data acquired inthe above step 65 indicates the operation of selecting an object andmoving the object to the virtual world displayed on the terminalapparatus 2 (step 66). Then, if the operation data indicates theoperation of selecting an object and moving the object, the CPU 5proceeds to step 67. If, on the other hand, the operation data does notindicate the operation of selecting an object and moving the object, theCPU 5 proceeds to step 71 (see FIG. 9).

In step 67, the CPU 5 sets objects on the basis of the operation dataacquired in the above step 65, and proceeds to the subsequent step. Forexample, the CPU 5 extracts the selection object selected by the user,from among the plurality of objects displayed on the monitor 4. Then,the CPU 5 changes the display position of the extracted selection objectto the inside of the virtual world displayed on the terminal apparatus 2(for example, to such a position that the selection object is displayedat the center of the LCD 11), and updates the display position data Db1of the selection object using the display position after the change.

Next, the CPU 5 sets performance streams to be employed for the firstsound (step 68), and proceeds to the subsequent step. For example, theCPU 5 deletes the stream number corresponding to the instrument to beplayed by the selection object selected in the above steps 65 and 66,from the stream numbers of the performance streams of which the audiooutputs are to be provided as the first sound. Then, the CPU 5 deletesthe performance parameters and the performance timing set for thedeleted stream number. Then, the CPU 5 updates the first sound data Dd.It should be noted that data representing the stream number, theperformance parameters, and the performance timing deleted from thefirst sound data Dd in accordance with the movement of the selectionobject will move to the second sound data De through the process of step69 described later.

Next, the CPU 5 sets performance streams to be employed for a secondsound (step 69), and proceeds to step 71 (see FIG. 9). For example, theCPU 5 sets the stream number corresponding to the instrument to beplayed by the selection object selected in the above steps 65 and 66(that is, the stream number deleted from the first sound data Dd inaccordance with the movement of the selection object), as the streamnumber of the performance stream of which the audio output is to beprovided as the second sound. Then, the CPU 5 stores data representingthese settings in the second sound data De. Further, the CPU 5 sets theperformance parameters of each performance stream for specifying thesteps, the tones, the volumes, and the like of the performance streamsof which the audio outputs are to be provided as the second sound, andthe performance timing of each performance stream, to the state of thesettings before the data has been deleted in the above step 68. Then,the CPU 5 stores data representing these settings in the second sounddata De.

Referring next to FIG. 9, in step 71, the CPU 5 determines whether ornot the operation data acquired in the above step 65 indicates aperformance operation on the selection object. Here, the performanceoperation is the operation of controlling the state, the action, and thelike of the selection object, and indicates, as a result, the operationof changing the volume, the tone, the step, the number of producedsounds, and the like of the sound of which the audio output is to beprovided from the loudspeaker 12 of the terminal apparatus 2. Then, ifthe operation data indicates a performance operation, the CPU 5 proceedsto step 72. If, on the other hand, the operation data does not indicatea performance operation, the CPU 5 proceeds to step 82.

In step 72, the CPU 5 determines whether or not the operation dataacquired in the above step 65 indicates the operation of changing theinstrument to be played by the selection object. Then, if the operationdata indicates the operation of changing the instrument, the CPU 5proceeds to step 73. If, on the other hand, the operation data does notindicate the operation of changing the instrument, the CPU 5 proceeds tostep 75.

In step 73, the CPU 5 sets objects on the basis of the operation dataacquired in the above step 65, and proceeds to the subsequent step. Forexample, on the basis of the operation data, the CPU 5 changes theinstrument to be played by the selection object, and updates theperformance instrument data Db2 of the selection object in accordancewith the instrument after the change.

Next, the CPU 5 sets performance streams to be employed for the secondsound (step 74), and proceeds to step 82. For example, on the basis ofthe change of instruments in the above step 73, the CPU 5 sets thestream number corresponding to the instrument after the change, as thestream number of the performance stream of which the audio output is tobe provided as the second sound, so as to replace the stream numbercorresponding to the instrument before the change. Then, the CPU 5updates the second sound data De. Further, the CPU 5 sets theperformance parameters of each performance stream for specifying thesteps, the tones, the volumes, and the like of the performance streamsof which the audio outputs are to be provided as the second sound, tothe default or to the settings before the change of instruments. Then,the CPU 5 also sets the performance timing of the performance streamcorresponding to the stream number after the change to the performancetiming set before the change of instruments. Then, the CPU 5 updates thesecond sound data De.

In step 75, the CPU 5 determines whether or not the operation dataacquired in the above step 65 indicates the operation of giving a userperformance. Then, if the operation data indicates the operation ofgiving a user performance, the CPU 5 proceeds to step 76. If, on theother hand, the operation data does not indicate the operation of givinga user performance, the CPU 5 proceeds to step 78.

In step 76, the CPU 5 sets objects on the basis of the operation dataacquired in the above step 65, and proceeds to the subsequent step. Forexample, on the basis of the operation data, the CPU 5 controls theaction of the selection object playing the instrument.

Next, the CPU 5 generates the second sound in accordance with a useroperation (step 77), and proceeds to step 82. As a first example, if thedifference between the time when the operation section 13 has beenoperated (for example, the time when the operation button has beenpressed) and the time when the production of the sound starts in theperformance stream corresponding to the instrument played by theselection object is within a predetermined time difference, the CPU 5sets a sound based on the performance stream as the second sound, andupdates the second sound data De. As a second example, the CPU 5 sets asound set in accordance with an operation on the operation section 13(for example, a sound having the step set at the position, in thevirtual world, overlapping a touch position on the touch panel, and alsohaving the timbre of the instrument played by the selection object), asthe second sound, and updates the second sound data De. It should benoted that in another example, the generation of the second sound basedon a user operation may be performed using another method.

In step 78, the CPU 5 determines whether or not the operation dataacquired in the above step 65 indicates the operation of changing thedevice for adding a sound effect to playing or singing. Then, if theoperation data indicates the user operation of changing the device, theCPU 5 proceeds to step 79. If, on the other hand, the operation datadoes not indicate the user operation of changing the device, the CPU 5proceeds to step 81.

In step 79, the CPU 5 sets objects on the basis of the operation dataacquired in the above step 65, and proceeds to the subsequent step. Forexample, the CPU 5 changes the device on the basis of the operationdata, and updates the performance instrument data Db2 of the selectionobject in accordance with the device after the change.

Next, the CPU 5 changes the volume, the tone, the step, and the like ofthe second sound in accordance with a user operation (step 80), andproceeds to step 82. For example, in accordance with the change ofperformance methods (the change of the device for adding a sound effectto playing or singing) made in the above step 79, the CPU 5 changes theperformance parameters for specifying the steps, the tones, the volumes,and the like of the performance streams of which the audio outputs areto be provided as the second sound. Then, the CPU 5 updates the secondsound data De using the performance parameters after the changes.

In step 81, the CPU 5 performs other processes on the basis of theoperation data acquired in the above step 65, and proceeds to step 82.Here, the other processes refer to processes different from theprocesses corresponding to the performance operation performed in theabove steps 72 through 80, and include the process of controlling thestate, the action, and the like of the selection object on the basis ofthe operation data, and changing the volume, the tone, the step, thenumber of produced sounds, and the like of the second sound. Then, alsoin the other processes, the CPU 5 updates the object data Db and thesecond sound data De on the basis of the operation data.

In step 82, the CPU 5 generates a terminal apparatus game image to bedisplayed on the terminal apparatus 2, and proceeds to the subsequentstep. For example, the CPU 5 reads, from the memory 6, data indicatingthe results of the game processing performed in the above steps 61through 81, and reads, from a VRAM (video RAM) or the like, datanecessary for generating a terminal apparatus game image. Then, the CPU5 generates a terminal apparatus game image using the read data, andstores the generated terminal apparatus game image in the VRAM. As anexample, the terminal apparatus game image may be generated as athree-dimensional CG image by: placing the objects in the virtual worldto be displayed on the terminal apparatus 2 on the basis of the objectdata Db; and calculating the virtual world viewed from a predeterminedvirtual camera.

Next, the CPU 5 generates a monitor game image to be displayed on themonitor 4 (step 83), and proceeds to the subsequent step. For example,the CPU 5 reads, from the memory 6, data indicating the results of thegame processing performed in the above steps 61 through 81, and reads,from the VRAM or the like, data necessary for generating a monitor gameimage. Then, the CPU 5 generates a monitor game image using the readdata, and stores the generated monitor game image in the VRAM. As anexample, the monitor game image may be generated as a three-dimensionalCG image by: placing the objects in the virtual world to be displayed onthe monitor 4 on the basis of the object data Db; and calculating thevirtual world viewed from a predetermined virtual camera.

Next, the CPU 5 generates a terminal apparatus game sound to be outputto the loudspeaker 12 of the terminal apparatus 2 (step 84), andproceeds to the subsequent step. For example, with reference to thesecond sound data De, the CPU 5 sets the stream numbers, the performanceparameters, and the performance timing of the performance streams ofwhich the audio outputs are to be provided. Then, on the basis of theset performance parameters and performance timing, the CPU 5superimposes the performance streams corresponding to the set streamnumbers as performance targets to generate a terminal apparatus gamesound. Specifically, on the basis of the performance parameters, the CPU5 adjusts the performance volumes, the performance tones, theperformance steps, and the like of the performance streams correspondingto the set stream numbers, and superimposes the performance streams togenerate a terminal apparatus game sound.

Next, the CPU 5 generates a monitor game sound to be output to theloudspeaker 41 of the monitor 4 (step 85), and proceeds to thesubsequent step. For example, with reference to the first sound data Dd,the CPU 5 sets the stream numbers, the performance parameters, and theperformance timing of the performance streams of which the audio outputsare to be provided. Then, on the basis of the set stream numbers,performance parameters, and performance timing, the CPU 5 superimposesthe performance streams similarly to the above step 84 to generate amonitor game sound.

Next, the CPU 5 transmits the terminal apparatus game image and theterminal apparatus game sound to the terminal apparatus 2 (step 86), andproceeds to the subsequent step. For example, the terminal apparatusgame image is received by the terminal apparatus 2, and is output to anddisplayed on the LCD 11. Further, the terminal apparatus game sound isreceived by the terminal apparatus 2, and is output from the loudspeaker12. It should be noted that the terminal apparatus game image may besubjected to a predetermined compression process when transmitted fromthe information processing apparatus 3 to the terminal apparatus 2. Inthis case, data of the terminal apparatus game image subjected to thecompression process is sent to the terminal apparatus 2, and issubjected to a predetermined decompression process by the terminalapparatus 2. Thereafter, the terminal apparatus game image is displayed.

Next, the CPU 5 outputs the monitor game image and the monitor gamesound to the monitor 4 (step 87), and proceeds to the subsequent step.For example, the monitor game image is acquired by the monitor 4, and isoutput to and displayed on the display screen of the monitor 4. Further,the monitor game sound is acquired by the monitor 4, and is output fromthe loudspeaker 41.

Next, the CPU 5 determines whether or not the game is to be ended (step88). Examples of conditions for ending the game include: thesatisfaction of the conditions under which the game is over or the gameis completed; and the fact that the user has performed the operation ofending the game. If the game is not to be ended, the CPU 5 returns tothe above step 65, and repeats the process thereof. If the game is to beended, the CPU 5 ends the processing indicated in the flow chart.Thereafter, the CPU 5 repeatedly performs a series of processes of steps65 through 88 until it is determined in step 88 that the game is to beended.

It should be noted that in the above exemplary embodiment, theinformation processing system 1 includes only one terminal apparatus 2.Alternatively, the information processing system 1 may be configured toinclude a plurality of terminal apparatuses 2. That is, the informationprocessing apparatus 3 may be capable of wirelessly communicating witheach of the plurality of terminal apparatuses 2, and may transmit imagedata and sound data to the terminal apparatus 2 and receive terminaloperation data from the terminal apparatus 2. Then, a selection objectselected by each terminal apparatus 2 may be displayed on the LCD 11 ofthe terminal apparatus 2, and a selection sound corresponding to theselection object may be output from the loudspeaker 12 of the terminalapparatus 2. When the information processing apparatus 3 wirelesslycommunicates with each of the plurality of terminal apparatuses 2, theinformation processing apparatus 3 may perform the wirelesscommunication with the terminal apparatus 2 in a time division manner orin a frequency division manner.

In addition, the terminal apparatus 2 described above functions as aso-called thin client terminal, which does not perform the series ofprocesses described with reference to FIGS. 8 and 9 or the informationprocessing performed by the information processing apparatus 3. Forexample, if information processing is executed by a plurality ofinformation processing apparatuses, it is necessary to synchronize theprocessing executed by the information processing apparatuses, whichcomplicates the processing. In contrast, as in the above exemplaryembodiment, if information processing is executed by one informationprocessing apparatus 3, and the terminal apparatus 2 receives anddisplays an image (that is, if the terminal apparatus 2 is a thin clientterminal), it is not necessary to synchronize the processing among aplurality of information processing apparatuses, which can simplify theprocessing. The terminal apparatus 2, however, may be an apparatushaving the function of performing predetermined information processingby a predetermined program (an information processing program), such asa handheld game apparatus. In this case, in the above exemplaryembodiment, at least some of the series of processes to be performed bythe information processing apparatus 3 may be performed by the terminalapparatus 2. As an example, if a game image is displayed on each of aplurality of terminal apparatus, using at least one terminal apparatuscapable of executing all of the series of processes described above, oneof the terminal apparatuses capable of executing all of the series ofprocesses serves as a main process execution apparatus that executes theseries of processes, so that the main process execution apparatustransmits game images based on the orientations of and operations on theother terminal apparatuses to the other terminal apparatuses. This makesit possible to output a similar game image to each of the terminalapparatuses and display the game image on the terminal apparatus.

It should be noted that the above descriptions are given using theexample where the information processing apparatus 3 performs theinformation processing processing. Alternatively, another apparatusother than the terminal apparatus 2 may perform at least some of theprocessing steps in the processing. For example, if the informationprocessing apparatus 3 is further configured to communicate with anotherapparatus other than the terminal apparatus 2 (for example, anotherserver, another game apparatus, or another mobile terminal), the otherapparatus may cooperate to perform the processing steps in theprocessing. As an example, the other apparatus may generate a virtualworld and a sound and perform the game processing using the virtualworld, and the result of the game processing may be displayed on theterminal apparatus 2 and the monitor 4. Another apparatus may thusperform at least some of the processing steps in the processing, whichenables processing similar to that described above. Further, theprocessing described above can be performed by a processor or thecooperation of a plurality of processors, the processor and theplurality of processors contained in an information processing systemincluding at least one information processing apparatus. Further, in theexemplary embodiment, the processing indicated in the flow chartdescribed above is performed by the CPU 5 of the information processingapparatus 3 executing a predetermined game program. Alternatively, someor all of the game processing indicated in the flow chart may beperformed by a dedicated circuit included in the information processingapparatus 3.

Here, the above variations make it possible to achieve the exemplaryembodiment also by a system form such as cloud computing, or a systemform such as a distributed wide area network or a local area network.For example, in a system form such as a distributed local area network,it is possible to execute the processing between a stationaryinformation processing apparatus (a stationary game apparatus) and ahandheld information processing apparatus (a handheld game apparatus) bythe cooperation of the apparatuses. It should be noted that, in thesesystem forms, there is no particular limitation on which apparatusperforms the process of each step of the processing described above.Thus, it goes without saying that it is possible to achieve theexemplary embodiment by sharing the processing in any manner.

In addition, the processing orders, the setting values, the conditionsused in the determinations, and the like that are used in the gameprocessing described above are merely illustrative. Thus, it goeswithout saying that the exemplary embodiment can be achieved also withother orders, other values, and other conditions.

In addition, the program may be supplied to the information processingapparatus 3 not only through an external storage medium such as theexternal memory 45, but also through a wired or wireless communicationlink. Further, the program may be stored in advance in a non-volatilestorage device included in the information processing apparatus 3. Itshould be noted that examples of an information storage medium havingstored therein the program may include CD-ROMs, DVDs, optical diskstorage media similar to these, flexible disks, hard disks,magneto-optical disks, and magnetic tapes, as well as non-volatilememories. Alternatively, an information storage medium having storedtherein the program may be a volatile memory for storing the program. Itcan be said that such a storage medium is a storage medium readable by acomputer or the like. For example, it is possible to provide the variousfunctions described above by causing a computer or the like to load aprogram from the storage medium and execute it.

The systems, devices and apparatuses described herein may include one ormore processors, which may be located in one place or distributed in avariety of places communicating via one or more networks. Suchprocessor(s) can, for example, use conventional 3D graphicstransformations, virtual camera and other techniques to provideappropriate images for display. By way of example and withoutlimitation, the processors can be any of: a processor that is part of oris a separate component co-located with the stationary display and whichcommunicates remotely (e.g., wirelessly) with the movable display; or aprocessor that is part of or is a separate component co-located with themovable display and communicates remotely (e.g., wirelessly) with thestationary display or associated equipment; or a distributed processingarrangement some of which is contained within the movable displayhousing and some of which is co-located with the stationary display, thedistributed portions communicating together via a connection such as awireless or wired network; or a processor(s) located remotely (e.g., inthe cloud) from both the stationary and movable displays andcommunicating with each of them via one or more network connections; orany combination or variation of the above.

The processors can be implemented using one or more general-purposeprocessors, one or more specialized graphics processors, or combinationsof these. These may be supplemented by specifically-designed ASICs(application specific integrated circuits) and/or logic circuitry. Inthe case of a distributed processor architecture or arrangement,appropriate data exchange and transmission protocols are used to providelow latency and maintain interactivity, as will be understood by thoseskilled in the art.

Similarly, program instructions, data and other information forimplementing the systems and methods described herein may be stored inone or more on-board and/or removable memory devices. Multiple memorydevices may be part of the same device or different devices, which areco-located or remotely located with respect to each other.

While some exemplary systems, exemplary methods, exemplary devices, andexemplary apparatuses have been described in detail above, the abovedescriptions are merely illustrative in all respects, and do not limitthe scope of the systems, the methods, the devices, and the apparatuses.It goes without saying that the systems, the methods, the devices, andthe apparatuses can be improved and modified in various manners withoutdeparting the spirit and scope of the appended claims. It is understoodthat the scope of the systems, the methods, the devices, and theapparatuses should be interpreted only by the scope of the appendedclaims. Further, it is understood that the specific descriptions of theexemplary embodiment enable a person skilled in the art to carry out anequivalent scope on the basis of the descriptions of the exemplaryembodiment and general technical knowledge. It should be understoodthat, when used in the specification, the components and the likedescribed in the singular with the word “a” or “an” preceding them donot exclude the plurals of the components. Furthermore, it should beunderstood that, unless otherwise stated, the terms used in thespecification are used in their common meanings in the field. Thus,unless otherwise defined, all the jargons and the technical terms usedin the specification have the same meanings as those generallyunderstood by a person skilled in the art in the field of the exemplaryembodiment. If there is a conflict, the specification (includingdefinitions) takes precedence.

As described above, the exemplary embodiment is useful as, for example,an information processing program, an information processing apparatus,an information processing system, and an information processing methodin order, for example, to obtain a wide variety of sound outputs.

What is claimed is:
 1. A non-transitory computer-readable storage mediumhaving stored therein an information processing program to be executedby a computer of an information processing apparatus that is coupled toa first display apparatus, a first loudspeaker, and another apparatushaving a second loudspeaker, the information processing program causingthe computer to execute: generating, as a first display image, an imageof at least a part of a virtual world; generating a first sound that isassociated with the virtual world; selecting, in accordance with a useroperation, a second sound that is less than all of the first sound;performing control such that the first display image is displayed on thefirst display apparatus; performing control such that the generatedfirst sound is output from the first loudspeaker in synchronization withthe first display image displayed on the first display apparatus; andperforming control such that the selected second sound is output, fromthe second loudspeaker, concurrently with the output of first sound andthe display of the first display image, where other parts of the firstsound not included in the second sound are not outputted by the secondloudspeaker, in the generation of the first sound, if the second soundhas been selected, the first sound is newly generated by removing thesecond sound from the first sound and the first sound that is outputfrom the first loudspeaker is the newly generated first sound.
 2. Thenon-transitory computer-readable storage medium having stored thereinthe information processing program according to claim 1, wherein in thegeneration of the first display image, an image in which a plurality ofobjects are placed in the virtual world is generated as the firstdisplay image, the information processing program further causing thecomputer to execute selecting at least one object from among theplurality of objects in accordance with a user operation, wherein in thegeneration of the first sound, the first sound is generated inaccordance with the objects displayed as the first display image; and inthe selection of the second sound, the second sound is selected inaccordance with the at least one selected object.
 3. The non-transitorycomputer-readable storage medium having stored therein the informationprocessing program according to claim 2, wherein in the generation ofthe first display image, an image obtained by removing the at least oneselected object from the plurality of objects and placing the at leastone remaining object is generated as the first display image; and in thegeneration of the first sound, a second sound based on the at least oneselected object is removed from the first sound based on the pluralityof objects to newly generate a sound based on the at least one remainingobject as the first sound and the newly generated sound is output fromthe first loudspeaker.
 4. The non-transitory computer-readable storagemedium having stored therein the information processing programaccording to claim 2, wherein the another apparatus is a second displayapparatus having a display screen, the information processing programfurther causing the computer to execute: generating an imagerepresenting the at least one selected object, as a second displayimage; and performing control such that the second display image isdisplayed on the second display apparatus, wherein in the selection ofthe second sound, the second sound is selected in accordance with the atleast one object displayed as the second display image.
 5. Thenon-transitory computer-readable storage medium having stored thereinthe information processing program according to claim 4, wherein in thegeneration of the first sound, a sound of which a sound source is the atleast one object displayed as the first display image is generated asthe first sound; and in the selection of the second sound, a sound ofwhich a sound source is the at least one object displayed as the seconddisplay image is selected as the second sound.
 6. The non-transitorycomputer-readable storage medium having stored therein the informationprocessing program according to claim 4, wherein the first displayapparatus is a stationary display apparatus; and the second displayapparatus is a portable display apparatus.
 7. The non-transitorycomputer-readable storage medium having stored therein the informationprocessing program according to claim 4, wherein the second displayapparatus includes an operation input unit configured to output databased on a user operation input, the information processing programfurther causing the computer to execute: acquiring the data output fromthe operation input unit; controlling, in accordance with the acquireddata, the at least one object displayed as the second display image; andchanging, in accordance with the acquired data, at least one of avolume, a tone, a step, and the number of produced sounds of the secondsound.
 8. The non-transitory computer-readable storage medium havingstored therein the information processing program according to claim 1,wherein in the generation of the first sound, a sound of performing onecomposition as an ensemble using a plurality of instruments in thevirtual world is generated as the first sound; in the selection of thesecond sound, a sound of performing the composition with at least oneinstrument selected from among the plurality of instruments inaccordance with a user operation is generated as the second sound; andin the control of outputting the second sound, the second sound ofperforming the composition is output from the second loudspeaker insynchronization with the performance of the composition output as thefirst sound from the first loudspeaker.
 9. The non-transitorycomputer-readable storage medium having stored therein the informationprocessing program according to claim 1, the information processingprogram further causing the computer to execute determining, inaccordance with a user operation, a position and/or a directionindicated in the virtual world displayed as the first display image,wherein in the generation of the first sound, an entire sound created inan entirety of the virtual world displayed as the first display image isgenerated as the first sound; and in the selection of the second sound,on the basis of the position and/or the direction indicated in thevirtual world, a sound created in a periphery of the position indicatedin the virtual world and/or a sound created in a predetermined rangewith respect to the direction indicated in the virtual world isgenerated and selected as the second sound.
 10. An informationprocessing apparatus coupled to a first display apparatus, a firstloudspeaker, and another apparatus that includes a second loudspeaker,the information processing apparatus comprising: a processing systemthat includes at least one hardware processor, the processing systemconfigured to: generate an image of a view of at least a part of avirtual world as a first display image; generate a first sound that iscreated in association with the virtual world; select, in accordancewith a user operation, a second sound that forms a part of the firstsound; output, to the first display apparatus, the first image; output,to the first loudspeaker and in synchronization with the first displayimage being displayed on the first display apparatus, audio that isbased on the generated first sound; and output, to the secondloudspeaker, the selected second sound concurrently with the first soundbeing output and the first display image being output, where other partsof the first sound not included in the second sound are not outputted bythe second loudspeaker, in the generation of the first sound, if thesecond sound has been selected, the first sound is newly generated byremoving the second sound from the first sound and the first sound thatis output from the first loudspeaker is the newly generated first sound.11. An information processing system comprising: a first displayapparatus; a first loudspeaker; a computing apparatus that includes asecond loudspeaker that is different from the first loudspeaker; and atleast one hardware processor that is configured to communicate with thecomputing apparatus, the at least one hardware processor configured to:generate a first display image of at least a part of a virtual world;generate a first sound that is associated with the virtual world;select, in accordance with a user operation, a second sound that forms apart of the first sound; based on selection of the second sound thatforms part of the first sound, regenerate the first sound by removingthe second sound from the first sound, perform control such that thefirst display image is displayed on the first display apparatus; performcontrol such that the regenerated first sound is output from the firstloudspeaker in synchronization with the first display image beingdisplayed on the first display apparatus; and perform control such thatthe second sound is output from the second loudspeaker concurrently withthe regenerated sound and the first display image, where other parts ofthe first sound not included in the second sound are not output by thesecond loudspeaker.
 12. An information processing method to be executedby a processing system that includes at least one hardware processorcoupled to a first display apparatus, a first loudspeaker, and anotherapparatus having a second loudspeaker, the information processing methodcomprising: generating a first display image of at least a part of avirtual world; generating a first sound that is associated with thevirtual world; selecting, in accordance with a user operation, a secondsound that is less than all of the first sound; performing control suchthat the first display image is displayed on the first displayapparatus; performing control such that the generated first sound isoutput from the first loudspeaker in synchronization with the firstdisplay image displayed on the first display apparatus; in accordancewith selection of the second sound, generating a third sound by removingthe second sound from the first sound and performing control such thatthe generated third sound is output from the first loudspeaker insynchronization with corresponding images displayed on the first displayapparatus; and performing control such that the selected second sound isoutput from the second loudspeaker concurrently with the output of thethird sound and the display of the corresponding images, where parts ofthe third sound that are not included in the second sound are notcontrolled to be outputted by the second loudspeaker.
 13. Theinformation processing system of claim 11, wherein the computingapparatus is a portable computing apparatus with the second loudspeakerintegrated into a body of the portable computing apparatus, the portablecomputing apparatus including a transceiver configured to wirelesslycommunicate with the at least one hardware processor to thereby receivesound data that is output as the second sound from the secondloudspeaker.
 14. The information processing apparatus of claim 10,wherein the another apparatus is a portable computing apparatus with thesecond loudspeaker integrated into a body of the portable computingapparatus, the portable computing apparatus including a transceiverconfigured to wirelessly communicate with the processing system tothereby receive sound data that is output as the second sound from thesecond loudspeaker.
 15. The non-transitory computer-readable storagemedium of claim 1, wherein the another apparatus is a mobile computingapparatus with the second loudspeaker integrated into a body of themobile computing apparatus, the mobile computing apparatus including atransceiver configured to wirelessly communicate with the informationprocessing apparatus to thereby receive sound data that is output as thesecond sound from the second loudspeaker.